DeepStory: Video Story QA by Deep Embedded Memory Networks

Question-answering (QA) on video contents is a significant challenge for achieving human-level intelligence as it involves both vision and language in real-world settings. Here we demonstrate the possibility of an AI agent performing video story QA by learning from a large amount of cartoon videos. We develop a video-story learning model, i.e. Deep Embedded Memory Networks (DEMN), to reconstruct stories from a joint scene-dialogue video stream using a latent embedding space of observed data. The video stories are stored in a long-term memory component. For a given question, an LSTM-based attention model uses the long-term memory to recall the best question-story-answer triplet by focusing on specific words containing key information. We trained the DEMN on a novel QA dataset of children's cartoon video series, Pororo. The dataset contains 16,066 scene-dialogue pairs of 20.5-hour videos, 27,328 fine-grained sentences for scene description, and 8,913 story-related QA pairs. Our experimental results show that the DEMN outperforms other QA models. This is mainly due to 1) the reconstruction of video stories in a scene-dialogue combined form that utilize the latent embedding and 2) attention. DEMN also achieved state-of-the-art results on the MovieQA benchmark.Comment: 7 pages, accepted for IJCAI 201

Examining Financial Anxiety Focusing on Interactions between Financial Knowledge and Financial Self-efficacy

This study examined whether the association between financial knowledge and financial anxiety depends on an individual’s financial self-efficacy by incorporating an interaction term between financial self-efficacy and financial knowledge. The self-efficacy component of the social cognitive theory of self-regulation has been tested using the 2018 National Financial Capability Study dataset. Households with higher financial knowledge and financial self-efficacy had lower levels of financial anxiety. After adding interaction terms of financial knowledge and financial self-efficacy in the model, the relationship between financial knowledge and financial anxiety depended on the levels of financial self-efficacy. Among those with anything less than high financial self-efficacy, the association between financial knowledge and financial anxiety weakens. The study found that financial knowledge and financial self-efficacy were significant in explaining financial anxiety and suggested implications for researchers, educators, and practitioners

Importance of Subjective Financial Knowledge and Perceived Credit Score in Payday Loan Use

This study examined the factors associated with consumers’ decisions to use payday loans. Using a sample of 24,201 respondents from the 2015 National Financial Capability Study (NFCS), structural equation modeling was used to analyze the relationships among the variables. The results indicated that payday loan use was associated with a series of consumers’ socio-psychological factors, including financial knowledge, perceived credit score, credit-card payment problems, and having emergency funds. The findings suggested that, to improve borrowing decisions and industry practices, discussions about consumers’ payday loan use and its underlying repayment problems should encompass policy intervention and institutional attention, rather than focusing on behavioral modification at the individual level alone

Multimedia Distribution Process Tracking for Android and iOS

The crime of illegally filming and distributing images or videos worldwide is increasing day by day. With the increasing penetration rate of smartphones, there has been a rise in crimes involving secretly taking pictures of people's bodies and distributing them through messengers. However, little research has been done on these related issue. The crime of distributing media using the world's popular messengers, WhatsApp and Telegram, is continuously increasing. It is also common to see criminals distributing illegal footage through various messengers to avoid being caught in the investigation network. As these crimes increase, there will continue to be a need for professional investigative personnel, and the time required for criminal investigations will continue to increase. In this paper, we propose a multimedia forensic method for tracking footprints by checking the media information that changes when images and videos shot with a smartphone are transmitted through instant messengers. We have selected 11 of the world's most popular instant messengers and two secure messengers. In addition, we selected the most widely used Android and iOS operating systems for smartphones. Through this study, we were able to confirm that it is possible to trace footprints related to the distribution of instant messengers by analyzing transmitted images and videos. Thus, it was possible to determine which messengers were used to distribute the video when it was transmitted through multiple messengers.Comment: 10 page

Analysis of online food purchasing behavior: a study of Sri Lankan consumers

Peer reviewedPublisher PD

Unsupervised Speech Representation Pooling Using Vector Quantization

With the advent of general-purpose speech representations from large-scale self-supervised models, applying a single model to multiple downstream tasks is becoming a de-facto approach. However, the pooling problem remains; the length of speech representations is inherently variable. The naive average pooling is often used, even though it ignores the characteristics of speech, such as differently lengthed phonemes. Hence, we design a novel pooling method to squash acoustically similar representations via vector quantization, which does not require additional training, unlike attention-based pooling. Further, we evaluate various unsupervised pooling methods on various self-supervised models. We gather diverse methods scattered around speech and text to evaluate on various tasks: keyword spotting, speaker identification, intent classification, and emotion recognition. Finally, we quantitatively and qualitatively analyze our method, comparing it with supervised pooling methods